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Abstract 

We study the structure of the set of equihbrium payoffs in fi- 
nite games, both for Nash equihbrium and correlated equihbrium. 
A nonempty subset of is shown to be the set of Nash equihbrium 
payoffs of a bimatrix game if and only if it is a finite union of rect- 
angles. Furthermore, we show that for any nonempty finite union of 
rectangles U and any polytope P C containing U, there exists a 
bimatrix game with U as set of Nash equilibrium payoffs and P as 
set of correlated equilibrium payoffs. The n-player case and the ro- 
bustness of this result to perturbation of the payoff matrices are also 
studied. 
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1 Introduction 



It is well known that the set of correlated equilibrium payoffs of a n-player 
finite game is a polytope in (Aumann, 1974). We show that, conversely, 
any polytope in M"^ is the set of correlated equilibrium payoffs of a n-player 
game. Moreover, for any n-player game G and any polytope P in R" con- 
taining the correlated equilibrium payoffs of G, there exists a n-player game 
with the same set of Nash equihbrium payoffs as G and P as set of correlated 
equilibrium payoffs. If G is a two-player game, it suffices to assume that P 
contains the Nash equilibrium payoffs of G (but not necessarily its correlated 
equilibrium payoffs). We also show that a subset of is the set of Nash 
equilibrium payoffs of a finite game if and only if it is a finite union of rectan- 
gles. Together with the previous result, this implies that: for any nonempty 
finite union of rectangles U and any polytope P containing [/, there exists 
a bimatrix game with U as set of Nash equilibrium payoffs and P as set of 
correlated equilibrium payoffs. 

Finally, we study the robustness of our results to perturbation of the 
payoff matrices. Since almost all games have a finite set of equilibria, all 
we can hope to show is that, for any finite set U in M" and any polytope 
P in containing U, there exists an open set of games whose set of Nash 
equilibrium payoffs is arbitrarily close to U and whose set of correlated equi- 
librium payoffs is arbitrarily close to P. We show that this is indeed the case. 

Notations and definitions: throughout, n denotes an integer weakly greater 
than 2. Let G be a n-player game with pure strategy set and payoff func- 
tion Ui : S := Xi<i<nSi ]R for player i. Let S^i := Xj^^Sj. A correlated 
equilibrium of G (Aumann, 1974) is a probability distribution n on the set 
of strategy profiles S* of G such that, for every player i in {1, .., n} and every 
pure strategy Si in Sf. 

Mti e Si, ^ //(sj, s_j) [ui{si, s_j) - Ui{ti, s_i)] > (1) 

S — i^S —i 

The set of correlated equilibria of G is a polytope, which contains the Nash 
equilibria. An extreme correlated equilibrium is an extreme point of this 
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polytope. 

Let Ui{ii) :— X]sg5 denote the average payoff of player i in the 

correlated equilibrium fi. The set of correlated equilibrium payoffs of G is 
the set of n-tuples ■u„(/i)) where is a correlated equilibrium of 

G. This is a polytope in M". It is denoted by GEP{G). Similarly, NEP{G) 
denotes the set of Nash equilibrium payoffs of G, and we may write NE 
and CE for "Nash equilibrium" and "correlated equilibrium", respectively. 
Finally, if A is a subset of M" then Conv{A) denotes its convex hull. 

2 Main results 

We begin with a result on correlated equilibria: 

Proposition 1 For any polytope P in R", there exists a n-player game 
whose set of correlated equilibrium payoffs (resp. convex hull of the set of 
Nash equilibrium payoffs) is P. 

Proof. We prove the result for two-player games; see the appendix for the 
n-player case. Let P be a polytope in R^. Let (a^i, yi),.--)(3^m) 2/m) be its 
extreme points. Assume that for every i in {1,2,.., m}, Xi and yi are strictly 
positive. This is without loss of generahty, since adding a constant to all 
payoffs of a game does not change the set of Nash equilibria and correlated 
equilibria. Consider the mxm game with payoff matrix (a^j, bij)i<ij<rn such 
that, for every i in {1, 2, m}, an — a^ni — Xi, bu — bim — yi, and all other 
payoffs are zero. For m — 4 this gives : 

0,0 0,0 
0,0 X2,y2 0,0 
0,0 0,0 xs,y3 

\ Xi,0 X2,0 X3,0 

Clearly, any diagonal square corresponds to a pure Nash equilibrium. Fur- 
thermore, choosing the last row (column) is a weakly dominant strategy for 
the row (column) player, and it gives a strictly higher payoff than choosing 



0,2/1 \ 

0,y2 
0,y3 



(2) 
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row (column) i ^ m whenever the column (row) player does not choose col- 
umn (row) i. It follows that in every correlated equilibrium, the probability 
of every off-diagonal square is zero, hence there are no other Nash equilibria 
or extreme correlated equilibria than the diagonal squares. Therefore, the 
convex hull of the set of Nash equilibrium payoffs coincides with the set of 
correlated equilibrium payoffs and is equal to P. ■ 

To state our next result, we first need a definition: let us say that a subset 
of is a rectangle if it is of the form [a, 6] x [c, d\ , for some real numbers a, 
6, c, (i, with a < b, c < d. 

Proposition 2 In any himatrix game, the set of Nash equilibrium payoffs 
is a finite union of rectangles. Conversely, for any nonempty finite union 
of rectangles U , there exists a himatrix game whose set of Nash equilibrium 
payoffs is U . Moreover, for any bimatrix game G, there exists a bimatrix 
game G' with NEP{G') = NEP{G) and CEP{G') = Gonv{NEP{G)). 

Proof. Consider a bimatrix game with pure strategy set / for player 1 and 
J for player 2. Let NEP C denote its set of Nash equilibrium payoffs. 
For any subsets /' of / and J' of J, let NEP{I', J') denote the set of payoffs 
of Nash equilibria with support I' x J'. For any subset ^4 of M^, let A denote 
the closure of A. We have 



where the second equahty holds because NEP is closed. Thus, to show that 
NEP is a finite union of rectangles, it suffices to show that NEP{r, J') is 
a rectangle. This follows easily from the fact that, in bimatrix games, Nash 
equilibria with the same support are exchangeable. 

Conversely, let m G N* and for 1 < i < m, let a^, bi, Ci, di be real numbers. 
Let U = Ui<j<m['^i' ^ [cijC^j]- Assuming w.l.o.g. that the numbers Oj, bi, 
Ci, di are all positive, we build below a bimatrix game with U as set of Nash 
equilibrium payoffs. Consider first the game with payoff matrices: 



NEP^ 



y NEP{I', J') = y NEP{I', J') 



rci,J'cJ rci,J'cJ 
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In this game, a player does not influence its own payoffs and the set of Nash 
equihbrium payoffs is [aj,6j] x [cj,(ij]. Let 

(A.,0)=f ^^'M , (0,5.) =f°'^' 

and consider the game built by blocks: 

.. (0,5i) 

(^2,^2) .. (0,52) 

.. (A^^uBm-l) iO,Bm-l) 

\ (v4i,0) (^2,0) .. (A„_i,0) {Am,Bm) 

where an isolated represents a 2 x 2 block of payoffs (0, 0). This game has 
the same structure as ([2]), but the payoffs and yi have been replaced by 
the blocks Ai and Bi, respectively. Any mixed strategy proflle with support 
in one of the blocks {Ai, Bi) is a Nash equilibrium. Furthermore, it is easy 
to prove along the lines of the proof of proposition [1] that there are no other 
Nash equilibria and that the set of correlated equilibria is the convex hull of 
the set of Nash equilibria. It follows that: flrst, the set of NE payoffs of ([3]) 
is equal to U, which shows that any flnite union of rectangles is the set of 
NE payoffs of a bimatrix game; and second, the set of CE payoffs of ([3]) is 
equal to Conv{U). Since the set of NE payoffs of a bimatrix game is always 
a flnite union of rectangles, this implies that for any bimatrix game G, there 
exists a bimatrix game G' (of type ([3])) such as NEP{G') = NEP{G) and 
CEP{G') = Gonv{NEP{G)). ■ 

Lemma 3 Let (xi, X2-, ■■■,Xn) G K"- For any n-player game G, there exists a 
n-player game with the same set of Nash equilibrium payoffs as G and whose 
set of correlated equilibrium payoffs is the convex hull of {xi,X2, ■■■,Xn) and 
of the set of correlated equilibrium payoffs of G. 

Proof. We prove the result in the two-player case. For the ra-player case, 
see the appendix. Let G be a two-player mi x m2 game and {x,y) E M^. 



(3) 
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Assume w.l.o.g. that x, y, and all the payoffs in G are positive. Consider the 
(3 + mi) X (3 + 1712) game 





0,0 


x + l,y-l 


x-l,y + l 




\ 




x-l,y + l 


0,0 


x + l,y-l 


[o,y] 






x + l,y-l 


x-l,y + l 


0,0 






v 




[x,0] 




G 


J 



where [x, 0] denotes a block of payoffs {x, 0) of appropriate size (same nota- 
tions for player 2). Let v denote the correlated strategy putting probability 
1/6 on every off-diagonal square of the top-left block, and probabihty on 
every other square of the whole payoff matrix. Clearly, 1/ is a correlated equi- 
librium of r, with payoff {x, y) , and every correlated equilibrium of G induces 
a correlated equilibrium of F. We claim that any correlated equilibrium of F 
is a convex combination of u and of a correlated equilibrium of G. 

To see this, let fi he a correlated equilibrium of F. Clearly, ^12 > /ii3, 
otherwise player 1 would have an incentive to deviate from his first to his last 
strategy (recall that all payoffs in G arc positive). Repeating this reasoning 
with other strategies and with player 2 leads to the chain of inequalities 

Since the first and last terms are equal, this is a chain of equalities, hence /i 
puts equal weight on all off-diagonal squares of the top-left block. It is then 
easy to see that puts probabihty zero on the diagonal of the top-left block 
as well as on the top-right and bottom-left blocks. This implies that /i is a 
convex combination of v and of a correlated equilibrium of G, proving the 
claim. 

It follows that : (i) a fortiori, any CE payoff of F is a convex combination 
of the payoff of u and of a correlated equilibrium payoff of G; and (ii) F and 
G have the same Nash equilibria, hence a fortiori the same NE payoffs. This 
concludes the proof. ■ 

Proposition 4 For any n-player game G and for any polytope P in R" con- 
taining GEP{G), there exists a game with the same set of Nash equilibrium 
payoffs as G and P as set of correlated equilibrium payoffs. 
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Proof. Let P be a polytope containing CEP{G), with q extreme points. 
Applying iteratively lemma [3] {q times) , we obtain a game with the same 
NE payoffs as G and whose set of CE payoffs is the convex hull of P and 
CEP{G), hence is equal to P. m 

In the two-player case, we get a stronger result: 

Corollary 5 For any himatrix game G and for any polytope P in R" con- 
taining NEP{G), there exists a game with the same set of Nash equilibrium 
payoffs as G and P as set of correlated equilibrium payoffs^ 

Proof. By proposition [2l there exists a game G' with NEP{G') = NEP{G) 
and CEP{G') = Gonv{NEP{G)). Applying proposition H to G' gives the 
result. ■ 

In light of proposition O corollary [5] may also be stated as follows : for 
any nonempty finite union of rectangles U in and for any polytope P in 

containing U, there exists a two-player game with U as set of Nash equi- 
librium payoffs and P as set of correlated equilibrium payoffs. Proposition 
E] and corollary [5] fully characterize the possible shapes of the set of Nash 
equilibrium payoffs and correlated equilibrium payoffs for two-player games. 
For n-player games, the general shape of the set of Nash equilibrium payoffs 
remains to be understood. Furthermore, we do not know whether, for any 
n-player game G, there exists a n-player game with the same set of Nash 
equilibrium payoffs as G and whose set of correlated equilibrium payoffs is 
the convex hull of its Nash equilibrium payoffs. 

3 Genericity 

The games used above are highly non generic. For instance, a small pertur- 
bation of the payoffs of (Ej) is enough to eliminate all its Nash and correlated 
equilibria but one. This raises the issue of the robustness of our results. Ide- 
ally, to show that, for instance, corollary [5] is robust, one would like to show 

^The difference with the n-player case in that we only assume P D NEP{G) instead 
of P D CEP{G). 
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that for any nonempty finite union of rectangles U and for any polytope P 
in containing U, there exists an open set of games whose set of Nash 
equihbrium payoffs is "close" to U and whose set of correlated equilibrium 
payoffs is "close" to P. This is hopeless however, since almost all games have 
a finite set of Nash equilibria. Thus, all we can hope to prove is the same 
result but with U being a finite set. This is the object of this section. 

We first need some definitions. Let e > 0. For any x in M", let ||x|| = 
maxi<j<„ \xi\. Let A and A' be subsets of M". Recall that A and A' are 
e-close in the Hausdorff distance sense if 

Vxe A,3x' e ||x-x'|| < e (5) 

and 

Vx' e A',3xG A, ||x-x'|| < e (6) 

Let G be a finite game with payoff function Uj for player i and let a > 0. 
The open ball of center G and radius a, denoted by B{G,a), is the set of 
all games G' with the same sets of players and strategies as in G and such 
that for every player i and every pure strategy profile s, \u'^{s) — Ui{s)\ < a, 
where u'^ is the payoff function of player i in G'. A set of games E is open 
if for every game G in S, S contains an open ball of center G and positive 
radius. 

Proposition 6 Let U be a finite set in M*^. Let P C MJ^ be a polytope 
containing U . There exists a (nonempty) open set of n-player gam,es whose 
set of Nash equilibrium payoffs is e-close to U and whose set of correlated 
equilibrium payoffs is e-close to P. 

Proof. We prove the result for two-player games. The proof for n-player 
games is similar (see the appendix). Let U — {{xi,yi), {xm,ym)}, let 
P C be a polytope containing U, with vertices {x[,y[),...,{x'g, y'^). Assume 
w.l.o.g. that, for all i in {1, Xi, yi, x[ and y[ are all positive. Let G^ 
denote the m x m game with payoff matrix {aij,bij)i<ij<m such that: for 
every i in {1, 2, m}, an — Xj and bn — yi, for every i in {1, 2, m — 1}, 
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Gn 



O'mi = Xi — a and foj^ = Hi — ct] and all other payoffs are zero. For m = 4 this 
gives: 

xi,yi 0,0 
0,0 X2,y2 0,0 
0,0 0,0 0:3,2/3 

\xi — a,0 X2 — a,0 X3 — a, 

Thus, Go is the game used in the proof of proposition [1] and NEP{Go) = U. 
For every {x,y) in M^, let C{x,y) denote the game corresponding to the 
top- left block of (g]). 



0,yi-a \ 
0, ?/2 - a 
0, 2/3 - a 

2^4, 2/4 / 



C{x,y) 



0,0 

X — l,y + 1 



\ x + l,y-l 
Finally, let denote the following game: 



F. 



x + l,y — l x — l,y + l\ 

0,0 x + l,y-l 
x-l,y + l 0,0 



(7) 



/ C{x[,y[) 








[0,y[-a] 





C{x'„y',) .. 





[0,2/^ -a] 








• c{x'^,y',) 


[0,2/;-«] 


\ [x[ — a, 0] 


[x2 — a, 0] .. 


■ K ~ 0] 


Ga 



J 

where [x[ — a, 0] means a block of payoffs {x[ — a, 0) of appropriate size. 

Fq is a slight modification of the game obtained from Go by iterative 
applications (g times) of lemma [21 Along the lines of the proof of lemma [31 it 
is easy to show that its Nash equilibria correspond to the Nash equilibria of 
Go and that its extreme correlated equilibria are: (i) its Nash equilibria; (ii) 
the probability distributions with support in one of the blocks G(x^,?/j') and 
putting probability 1/6 on every off-diagonal square of this block. It follows 
that A^^P(Fo) = U and GEP{To) = P. 

Moreover, for any a small enough and any game F in i?(FQ,,a/2), every 
Nash equilibrium of Fo is a Nash equilibrium of F; therefore. 



\/{ao,bo) e NEP{To),3{a,b) e NEP{T),\\{a,b) - {ao,bo)\\ <3a/2 (8) 
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(a closer look shows that we may replace 3a/2 by ajl in the RHS, but 
this is not needed). Finally, let e > 0. By upper semi-continuity of the 
Nash equilibrium correspondence, for a small enough, for any game F in 
5(Fo,3a/2), 

V(a,6) eiVEP(F),3(ao,M eiV^P(Fo),||(ao,6o)-(a,6)|| <e (9) 

It follows from ([8]) and ([9]) that, for any a small enough and any game F in 
i?(FQ,,a/2), NEP{r) and NEP^Tq) are e-close. The same argument (up to 
replacement of Nash equilibrium by correlated equilibrium everywhere) shows 
that for every a small enough and for every game F in -B(Fq,, a/2), CEP{T) 
and CEP{Tq) are e-close. Recalling that NEP{Tq) = U and CEP{To) = P, 
this completes the proof. ■ 

Note that, in the above proof, for every game F in P(Fq,, a/ 2), any Nash 
equilibrium (resp. extreme correlated equilibrium) of Fq is actually a strict 
Nash equilibrium (resp. strict correlated equilibriurr§]) of F. Since further- 
more a convex combination of strict correlated equilibria is a strict correlated 
equilibrium, it follows that for every finite set U in R"^, every polytope P con- 
taining U and every e > 0, there exists an open set of games F such that: 
first, both the set of Nash equilibrium payoffs and the set of strict Nash 
equilibrium payoffs of F are e-close to U; second, both the set of correlated 
equilibrium payoffs and the set of strict correlated equilibrium payoffs of F 
are e-close to P. 

A Proofs of lemma [3] and propositions [I and 
M in the n-player case. 

Proof of lemma [3]. Let G be a three-player game and let {x,y,z) G M'^. 
Assume w.l.o.g. that x, y, z, and all the payoffs in G are positive. Let 
C{x, y, z) denote the 3x3x1 game (player 3 is a dummy) where the payoffs 

correlated equilibrium /i is strict if for every pure strategy Si with positive marginal 
probability under /j,, the inequalities in ^ are strict. 
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of players 1 and 2 are as in ([7j) and the payoff of player 3 is always z. 
( 0,0, z 

yx + l,?/ — 1,2; X 



x + l,y — l,z x — l,y+l,z\ 

0. 0, z X + l,y — 1, z 

1, y + l,z 0,0, z 



(10) 



Let 



C{x,y,z) 


[0,2/, 0] \ 


( [0,0,;^] 


[0,2/,^] 


[x,0,0] 


[x,y,o] J 


\ k,0,2] 


G 



where [x, 0, 0] denotes a block of payoffs {x, 0, 0) of appropriate size. The 
same proof as in the two-player case shows that F has the same set of NE 
payoffs as G and that its set of CE payoffs is the convex hull of (x, y, z) and 
of the set of CE payoffs of G. 

In the n-player case, the generalization of G{x,y,z) simply consists in 
adding more dummy players with constant payoff. The generalization of F 
should be clear from the following description of the three-player case: the 
pure strategy set of player i is S'^ U Si (disjoint union), where Si and S'^ are 
respectively the pure strategy set of player i in G and in G{x,y,z). If for 
every i, player i chooses a strategy in Si (resp. S'i), then the payoffs are as in 
G (resp. as in G{x,y,z)). If there exist players i and j such that i chooses 
a strategy in Si and j a strategy in S'p then the payoff of player 1 (resp. 2, 
3) is X (resp. y, z) if he chooses a strategy in S'i (resp. 5*2, 5*3) and otherwise. 



Proof of propositions [T] and [6l Let U = {{xi, yi, Zi),l < i < m} and 

let P be the convex hull of {{x'i, y'i, z'^) , I <i < q}. Assume w.l.o.g. that the 
numbers Xi, x'i, etc. are all positive. Let 



Gr 



j xi,yi,zi 


0,0, 





0,0 







0,2/1 -a,0 


0,0,0 


2^2,2/2 


Z2 ■ 


0,0 







0,2/2 -a,0 


0,0,0 


0,0, 





•^m—ly ym- 


-1, 


^m—l 


0,2/m-i - a,0 


\ xi — a, 0, 


X2 — cy 


0,0 . 


•^m—l 


a. 


0,0 


•^mi ymi Z'm 



The same argument as in the two-player case shows that the Nash equilibria 
of Go are equal to its extreme correlated equilibria and correspond to the 
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diagonal squares. It follows that NEP{Gq) = U and that CEP{Gq) is the 
convex hull of U. This proves proposition [T] in the three-player case (for the 
n-player case, just add more dummy players). 

Now recall (fTOj) and let G' denote the (3g + m) x (3g + m) x 1 game with 
block diagonal payoff matrix 



G' 



\ 







\ 



Hqi '■^q) 

Let X, z be positive real numbers. Let Fq denote the (3g+m) x (3g+m) x 2 
game: 



a 


[0,1/: -«,o] ^ 


[ [0,0,z:-a] 




\x] - a, 0, 0] 




V [x,0,z] 


GoL 



This should be read as follows: if player 1 chooses row i > 3g and player 
2 chooses column j < 3g with 3p + 1 < j < 3p + 3, then the payoffs are 
0, 0) if player 3 chooses the left matrix and (x, 0, if player 3 chooses 
the matrix on the right. 

Fix e > and assume that P D U. The same arguments as in the 
two-player case show that, for every a small enough, and for every game F 
in B{Ga,a/2), NEPiT) is e-close to U and CEP(T) is e-close to P. The 
n-player case is similar^ 
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